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(54) Method of updating reference background image, method of detecting entering objects and 
system for detecting entering objects using the methods 



(57) A method of updating a reference background 
image used for detecting objects entering an image 
pickup view field based on the binary image generated 
from the difference between an input image and the ref- 
erence background image of the input image. The 
image pickup view field is divided into a plurality of view 
field areas (201), and a reference background image 
corresponding to each of the fixed divided view field 
areas is updated (204). An entering object detection 
apparatus using this method has an input image 
processing unit including an image memory (603) stor- 
ing an input image from an image input unit, a program 
memory (606) storing the program for activating the 
entering object detecting unit, a work memory (604) and 
a central processing unit (605) activating the entering 
object detecting unit in accordance with the program. 
The processing unit has an entering object detecting 
unit (104) determining intensity difference for each pixel 
between the input image and the reference background 
image not including an entering object to be detected 
and detecting an area with the difference larger than a 
predetermined threshold as an entering object, a divid- 
ing unit (201) dividing the image pickup view field of the 
image input unit into a plurality of view field areas, an 
image change detecting unit (202) detecting the change 
of the image in each of the divided view field areas, and 
a reference background image updating unit (204) 
updating each portion of the reference background 



image corresponding to each of the divided view field 
areas associated with the portion of the input image free 
of the image change, wherein the entering object 
detecting unit detects entering objects based on the 
updated reference background image. 
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Description 

BACKGROUND OF THE INVENTION 

5 [0001] The present invention relates to a monitoring system, or more in particular to an entering object detecting 
method and an entering object detecting system for automatically detecting persons who have entered the image 
pickup view field or vehicles moving in the image pickup view field from an image signal. 

[0002] An image monitoring system using an image pickup device such as a camera has conventionally been widely 
used. In recent years, however, demand has arisen for an object tracking and monitoring apparatus for an image mon- 
10 itoring system by which objects such as persons or automobiles (vehicles) entering the monitoring view field are 
detected from an input image signal and predetermined information or alarm is produced automatically without any per- 
son viewing the image displayed on a monitor. 

[0003] For realizing the object tracking and monitoring system described above, the input image obtained from an 
image pickup device is compared with a reference background image, i.e. an image not including an entering object to 
75 be detected thereby to detect a difference in intensity (or brightness) value for each pixel, and an area with a large inten- 
sity difference is detected as an entering object. This method is called a subtraction method and has found wide appli- 
cations. 

[0004] In this method, however, a reference background image not including an entering object to be detected is 
required, and in the case where the brightness (intensity value) of the input image changes due to the illuminance 
20 change in the monitoring view field, tor example, the reference background image is required to be updated in accord- 
ance with the illuminance change. 

SUMMARY OF THE INVENTION 

25 [0005] Several methods are available for updating a reference background image. They include a method for produc- 
ing a reference background using an average value of the intensity for each pixel of input images in a plurality of frames 
(called the averaging method), a method for sequentially producing a new reference background image from the 
weighted average of the present input image and the present reference background image, calculated under a prede- 
termined weight (called the add-up method), a method in which the median value (central value) of temporal change of 

30 the intensity of a given pixel having an input image is determined as a background pixel intensity value of the pixel and 
this process is executed for all the pixels in a monitoring area (called the median method), and a method in which the 
reference background image is updated for pixels other than in the area entered by an object and detected by the sub- 
traction method (called the dynamic area updating method). 

[0006] In the averaging method, the add-up method and the median method, however, many frames are required for 
35 producing a reference background image, and a long time lag occurs before complete updating of the reference back- 
ground image after an input image change, if any. In addition, an image storage memory of a large capacity is required 
for an object tracking and monitoring system. In the dynamic area updating method, on the other hand, a intensity mis- 
match occurs in the boundary between pixels with the reference background image updated and pixels with the refer- 
ence background image not updated in the monitoring view field. Here, the mismatch refers to a phenomenon that it 
40 falsely looks as if a contour exists at a portion where the background image has in fact a smooth change in intensity 
due to generation of a stepwise intensity change at an interface between updated pixels and those not updated. For 
specifying the position where the mismatch has occurred, the past images of detected entering objects are required to 
be stored, so that an image storage memory of a large capacity is required for the object tracking and monitoring sys- 
tem. 

45 [0007] An object of the present invention is to obviate the disadvantages described above and to provide a highly reli- 
able method and a highly reliable system for updating a background image. 

[0008] Another object of the invention is to provide a method and a system capable of rapidly updating the background 
image in accordance with the brightness or intensity (intensity value) change of an input image using an image memory 
of a small capacity. 

so [0009] Still another object of the invention is to provide a method and a system for updating the background image in 
which an intensity mismatch which may occur between the pixels updated and the pixels not updated of the reference 
background image has no effect on the reliability for detection of an entering object. 

[0010] A further object of the invention is to provide a method and a system for detecting entering objects high in 
detection reliability. 

55 [001 1] In order to achieve the objects described above, according to one aspect of the invention, there is provided a 
reference background image updating method in which the image pickup view field is divided into a plurality of areas 
and the portion of the reference background corresponding to each divided area is updated. 
[0012] The image pickup view field may be divided and the reference background image for each divided area may 
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be updated after detecting an entering object. Alternatively, after dividing the image pickup view field, an entering object 
may be detected for each divided view field and the corresponding portion of the reference background image may be 
updated. 

[001 3] Each portion of the reference background image is updated in the case where no change indicating an entering 
object exists in the corresponding input image from an image pickup device. 

[001 4J Preferably, the image pickup view field is divided by one or a plurality of boundary lines substantially parallel 
to the direction of movement of an entering object. 

[001 5] Preferably, the image pickup view field is divided by an average movement range of an entering object during 
each predetermined unit time. 

[0016] Preferably, the image pickup view field is divided by one or a plurality of boundary lines substantially parallel 
to the direction of movement of an entering object and the divided view field is subdivided by an average movement 
range of an entering object during each predetermined unit time. 

[0017] According to an embodiment, the entering object includes an automobile, the input image includes a vehicle 
lane, and preferably, the image pickup view field is divided by one or a plurality of lane boundaries 
[0018] According to another embodiment, the entering object is an automobile, the input image includes a lane, and 
preferably, the image pickup view field is divided by an average movement range of the automobile during each prede- 
termined unit time. 

[0019] According to still another embodiment, the entering object is an automobile, the input image includes a lane, 
and preferably the image pickup view field is divided by one or a plurality of lane boundaries, and the divided image 
pickup view field is subdivided by an average movement range of the automobile during each predetermined unit time. 
[0020] According to a further embodiment the reference background image can be updated within a shorter time 
using the update rate of 1/4, for example, than by the add-up method generally using the lower update rate of 1/64. 
[0021 ] According to another aspect of the invention, there is provided a reference background image updating system 
used for detection of entering objects in the image pickup view field based on a binarized image generated from the dif- 
ference between an input image and and the reference background image of the input image, comprising a dividing unit 
for dividing the image pickup view field into a plurality of view areas and an update unit for updating the reference back- 
ground image corresponding to each of the divided view fields independently for each of the divided view fields. 
[0022] According to still another aspect of the invention, there is provided an entering object detecting system com- 
prising an image input unit, a processing unit for processing the input image including an image memory for storing an 
input image from the image input unit, a program memory for storing the program for the operating the entering object 
detecting system and a central processing unit for activating the entering object detecting system in accordance with 
the program, wherein the processing unit includes an entering object detecting unit for determining the intensity differ- 
ence for each pixel between the input image from the image input unit and the reference background image not includ- 
ing the entering object to be detected and detecting the binarized image generated from the difference value, i.e. 
detecting the area where the difference value is larger than a predetermined threshold as an entering object, a dividing 
unit for dividing the image pickup view field of the image input unit into a plurality of view field areas, an image change 
detecting unit for detecting the image change in each divided view field area, and a reference background image update 
unit for updating each portion of the reference background image corresponding to the divided view field area associ- 
ated with to the portion of the input image having no image change, wherein the entering object detecting unit detects 
an entering object based on the updated reference background image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0023] 

Fig. 1 is a flowchart for explaining the process of updating a reference background image and executing the proc- 
ess for detecting an entering object according to an embodiment of the invention. 

Fig. 2 is a flowchart for explaining the process of updating a reference background image and executing the proc- 
ess for detecting an entering object according to another embodiment of the invention. 
Figs. 3A, 3B are diagrams useful for explaining an example of dividing the view field according to the invention. 
Figs. 4A, 4B are diagrams useful for explaining an example of dividing the view field according to the invention. 
Fig. 5 is a diagram for explaining an example of an image change detecting method. 
Fig. 6 is a block diagram showing a hardware configuration according to an embodiment of the invention. 
Fig. 7 is a diagram for explaining the principle of object detection by the subtraction method. 
Fig. 8 is a diagram for explaining the principle of updating a reference background image by the add-up method. 
Fig. 9 is a diagram for explaining the intensity change of a given pixel over N frames. 

Fig. 10 is a diagram for explaining the principle of updating the reference background image by the median method. 
Figs. 1 1 A to 1 1 C are diagrams useful for explaining the view field dividing method of Figs. 3A, 3B in detail. 
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Figs. 12A to 12C are diagrams useful tor explaining the view field dividing method of Figs. 4A, 4B in detail. 
DESCRIPTION OF THE EMBODIMENTS 

5 [0024] First, the processing by the subtraction method will be explained with reference to Fig. 7. Fig. 7 is a diagram 
for explaining the principle of object detection by the subtraction method, in which reference numeral 701 designates 
an input image f, numeral 702 a reference background image r, numeral 703 a difference image, numeral 704 a bina- 
rized image, numeral 705 an image of an object detected by the subtraction method and numeral 721 a subtracter. In 
Fig. 17, the subtracter 721 produces the difference image 703 by calculating the intensity difference for each pixel 

10 between the input image 701 and the reference background image 702 prepared in advance. Then, the intensity of the 
pixels of the difference image 703 less than a predetermined threshold is defined as "0" and the intensity of the pixels 
not less than the threshold as "255" (the brightness of each pixel is calculated in 8 bits) thereby to produce the binarized 
image 704. As a result, the human object included in the input image 701 is detected as an image 705 in the binarized 
image 704. The reference background image is a background image not including any entering object to be detected. 

is In the case where the intensity (intensity or brightness value) of the input image changes due to the change in illumi- 
nance or the like in the monitoring area, the reference background image is required to be updated in accordance with 
the illuminance change. The methods for updating the reference background image, namely, the areraging method, the 
add-up method, the median method and the dynamic area updating method will be briefly explained below. 
[0025] First, the averaging method will be explained. This method averages images of a predetermined number of 

20 frames pixel by pixel to generate an updated background image. In this method, however, in order to obtain an accurate 
background image, it is necessary that the number of the frames to be used for averaging may be quite large, for exam- 
ple, 60 (corresponding to the period of 10 seconds supposing 6 frames per second). Therefore a large time lag (about 
10 seconds) is unfavorably generated between the time at which images for reference background image generation 
are inputted and the time at which subtraction processing for object detection is executed. Due to this time lag, a prob- 

25 lem arises such that it becomes impossible to obtain a reference background image which is accurate enough to be 
usable as a current background image for object detection in such cases as when the brightness of the imaging view 
field suddenly changes when the sun is quickly blocked by clouds or when the sun is quickly getting out of the clouds. 
[0026] Next, the add-up method will be explained with reference to Fig. 8. Fig. 8 is a diagram for explaining a method 
of updating the reference background image using the add-up method, in which numeral 801 designates a reference 

30 background image, numeral 802 an input image, numeral 803 a reference background image, numeral 804 an update 
rate, numerals 805, 806 posters, numeral 807 an entering object, and numeral 821 is a weighted average calculator. In 
the add-up method, the weighted average of the present reference background image 801 is calculated with a prede- 
termined weight (update rate 804) imposed on the present input image 802 thereby to produce a new reference back- 
ground image 803 sequentially. This process is expressed by equation (1) below. 

35 

'to+A y) - 0- R ) x r t0 + R x f l0 (x, y) (1) 

where r t0+1 is a new reference background image 803 used at time point to+1, r^ a reference background image 801 at 
time point to, f^ an input image 802 at time point to and R an update rate 804. Also, (x, y) is a coordinate indicating the 

40 pixel position. In the case where the background has changed such as by attaching the poster 805 anew in the input 
image 802, for example, the reference background image is updated in the new reference background image 803 such 
as by the poster 806. When the update rate 804 is increased, the reference background image 803 is also updated 
within a short time against the background change of the input image 802. In the case where the update rate 804 is set 
to a large value, however, the image of an entering object 807, if any is present in the input image, is absorbed into the 

45 new reference background image 803 in the input image. Therefore, the update rate 805 is required to be empirically 
set to a value (1/64, 1/32, 3/64, etc. for example) at which the image of the entering object 807 is not absorbed into the 
new reference background image 803. In the case where the update rate is set to 1/64, for example, it is equivalent to 
producing the reference background image by the averaging method using the average intensity value of an input 
image of 64 frames for each pixel. In the case where the update rate is set to 1/64, however, the update process for 64 

so frames is required from the time of occurrence of a change in the input image to the time when the change is entirely 
reflected in the reference background image. This means that the time as long as ten and several seconds is required 
before complete updating in view of the normal fact that about five frames are used per second for detecting an entering 
object. An example of the object recognition system using the add-up method described above is disclosed in JP-A-1 1- 
1 75735 published on July 2. 1 999 (Japanese Patent Application No. 9-344912, filed December 1 5, 1997). 

55 [0027] Now, the median method will be explained with reference to Figs. 9 and 10. Fig. 9 is a graph indicating the 
intensity value with time of an input image of predetermined N frames (N: natural number) for a given pixel, in which the 
horizontal axis represents the time and the vertical axis the intensity value, and numeral 903 designates the intensity 
value data of the input image of N frames arranged in temporal order. Fig. 10 is a diagram in which the intensity data 
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obtained in Fig. 9 are arranged in the order of magnitude along the time axis, in which the horizontal axis represents 
the number of frames and the vertical axis the intensity value, numeral 904 the intensity value data with the intensity 
value arranged in the ascending order of magnitude and numeral 905 the median value. 

[0028] In the median method, as shown in Fig. 9, the intensity data 903 is obtained from an input image for the same 
5 pixel of predetermined N frames. Then, as shown in Fig. 10, the intensity data 903 are arranged in the ascending order 
to produce the intensity data 904, so that the intensity value 905 for N/2 (median value) is defined as the intensity of a 
reference background pixel. This process is executed for ail the pixels in the monitoring area. This method is expressed 
as 

10 r t0+1 (x,y) = med{f 1 (x,y)} (2) 

t 0 £t<t 0 -N 

where r t0+1 is a new reference background image 905 used at time point to+ 1 » R to a reference background image at 
is time point to, f t0 an input image at time point to, and med {} the median calculation process. Also, (x, y) is the coordinate 

indicating the pixel position. Further, the number of frames required for the background image production is set to about 

not less than twice the number of frames in which an entering object of standard size to be detected passes one pixel. 

In the case where an entering object passes a pixel in ten frames, for example, N is set to 20. The intensity value, which 

is arranged in the ascending order of magnitude in the example of the median method described above, can alterna- 
te? tiveiy be arranged in the descending order. 

[0029] The median method has the advantage that the number of frames of the input image required for updating the 

reference background image can be reduced. 

[0030] Nevertheless, as many image memories as N frames are required, and the brightness values are required to 
be rearranged in the ascending or descending order for median calculations. Therefore, the calculation cost and the cal- 
25 culation time are increased. An example of an object detecting system using the median method described above is 
disclosed in JP-A-9-73541 (corresponding to U.S. Serial No. 08/64601 8 filed on May 7. 1996 and EP 96303303.3 filed 
on May 13, 1996). 

[0031] Finally, the dynamic area updating method will be explained. This method, in which the entering object area 
705 is detected by the subtraction method as shown in Fig. 7, and the reference background image 702 is updated by 
30 the add-up method for the pixels other than the detected entering object area 705, is expressed by equation (3) below. 

3(x,y)e{(x,y)|d t0 (x,y) = 0} (3) 

r to+i( x . y) - o - R1 ) x r to + R ' x f «>(*■ y) 

35 

where d^ is a detected entering object image 704 at time point to, and the intensity value of the pixels having the enter- 
ing object therein are set to 255 and the intensity values of other pixels are set to 0. Also, r,o +1 indicates a new reference 
background image 803 used at time point to+1 , r^ a reference background image 801 at time point to, fto an input image 
802 at time point to. and R' an update rate 804. Further, (x, y) represents the coordinate indicating the position of a given 
40 pixel. 

[0032] In this dynamic area updating method, the update rate R' can be increased as compared with the update rate 
R for the add-up method described above. As compared with the add-up method, therefore, the time can be shortened 
from when the input image undergoes a change until when the change is updated in the reference background image. 
In this method, however, updated pixels coexist with pixels not updated in the reference background image, and there- 
45 fore in the case where the illuminance changes in the view field, the mismatch of the intensity value is caused. 

[0033] Assume, for example, that the intensity value A of the pixel a changes to the intensity value A* and the intensity 
value B of an adjacent pixel b changes to the intensity value B\ The pixel a having no entering object is updated toward 
the intensity value A' following the particular change. With the pixel b having the entering object, however, the intensity 
value is not updated and remains at B. In the event that adjacent two pixels a and b have substantially the same inten- 
se shy value, therefore, the presence of a pixel updated and a pixel not updated as in the above-mentioned case causes 
the mismatch of the intensity value. 

[0034] This mismatch is developed in the boundary portion of the entering object area 705. Also, this mismatch 
remains unremoved until the complete updating of the reference background image after the entering object passes. 
Even after the passage of the entering object, therefore, the mismatch of the intensity value remains unremoved, 
55 thereby leading to the inaccuracy of detection of a new entering object. For preventing this inconveniencej.e. for spec- 
ifying the point of mismatch to update the reference background image sequentially, it is necessary to hold as many 
detected entering object images as the frames required for updating the reference background image. 
[0035] An example of an object detecting system using the dynamic area updating method described above is dis- 
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closed in JP-A-1 1-127430 published on May 1 1 , 1999 (Japanese Patent Application No. 9-291910 filed on October 24, 
1997). 

[0036] Now, embodiments of the present invention will be described with reference to the drawings. 

[0037] A configuration of an object tracking and monitoring system according to an embodiment will be explained. Fig. 

5 6 is a block diagram showing an example of a hardware configuration of an object tracking and monitoring system. In 
Fig. 6, numeral 601 designates an image pickup device such as a television (TV) camera, numeral 602 an image input 
interface (l/F), numeral 609 a data bus, numeral 603 an image memory, numeral 604 a work memory, numeral 605 a 
CPU, numeral 606 a program memory, numeral 607 an output interface (l/F), numeral 608 an image output l/F, numeral 
610 an alarm lamp, and numeral 611 a surveillance monitor. The TV camera 601 is connected to the image input l/F 

10 602, the alarm lamp 610 is connected to the output l/F 607, and the monitor 61 1 is connected to the image output l/F 
608. The image input l/F 602, the image memory 603, the work memory 604, the CPU 605, the program memory 606, 
the output l/F 607 and the image output l/F 608 are connected to the data bus 609. In Fig. 6, the TV camera 601 picks 
up an image in the image pickup view field including the area to be monitored. The TV camera 601 converts the image 
thus picked up into an image signal. This image signal is input to the image input l/F 602. The image input l/F 602 con- 

75 verts the input image signal into a format for processing in the object tracking system, and sends it to the image memory 
603 through the data bus 609. The image memory 603 stores the image data sent thereto. The CPU 605 analyzes the 
images stored in the image memory 603 through the work memory 604 in accordance with the program held in the pro- 
gram memory 606. As a result of this analysis, information is obtained as to whether an object has entered a predeter- 
mined monitoring area (for example, the neighborhood of a gate along a road included in the image pickup view field) 

20 in the image pickup view field of the TV camera. The CPU 605 turns on the alarm lamp 610 through the output l/F 607 
from the data bus 609 in accordance with the processing result, and displays an image of the processing result, for 
example, on the monitor 61 1 through the image output l/F 608. The output l/F 607 converts the signal from the CPU 
605 into a format usable by the alarm lamp 610, and sends it to the alarm lamp 610. The image output l/F 608 converts 
the signal from the CPU 605 into a format usable by the monitor 61 1 , and sends it to the alarm lamp 610. The monitor 

25 61 1 displays an image indicating the result of detecting an entering object. The image memory 603, the CPU 605, the 
work memory 604 and the program memory 606 make up an input image processing unit. All the flowcharts below will 
be explained with reference to an example of the hardware configuration of the object tracking and monitoring system 
described above. 

[0038] Fig. 1 is a flowchart for explaining the process of updating the reference background image and detecting an 

30 entering object according to an embodiment of the invention. The process of steps 101 to 106 in the flowchart of Fig. 1 
will be explained below with reference to Fig. 7 which has been used for explaining the prior art. 
[0039] At time point to. an input image 701 shown in Fig. 7 corresponding to 320 x 240 pixels is produced from a TV 
camera 601 (image input step 101). Then, the difference in intensity for each pixel between the input image 701 and 
the reference background image 702 stored in the image memory 603 is calculated by a subtracter 721 thereby to pro- 

35 duce a difference image 703 (difference processing step 1 02). The difference image 703 is processed with a threshold. 
Specifically, the intensity value of a pixel not less than a preset threshold value is converted into "255" so that the par- 
ticular pixel is set as a portion where a detected object exists, while the intensity value less than the threshold value is 
converted into "0" si that the particular pixel is defined as a portion where no detected object exists, thereby producing 
a binarized image 704 (binarization processing step 103). The preset threshold value is the one for determining the 

40 presence or absence of an entering object with respect to the difference value between the input image and the refer- 
ence background image and set at such a value that the entering object is not buried in a noise or the like as a result of 
binarization. This value is dependent on the object to be monitored and set experimentally. According to an example of 
the embodiment of the invention, the threshold value is set to 20. As an alternative, the threshold value may be varied 
in accordance with the difference image 703 obtained by the difference processing. 

45 [0040] Further, a mass of area 705 where the brightness value is "255" is extracted by the well-known labeling method 
and detected as an entering object (entering object detection processing step 104). In the case where no entering 
object is detected in the entering object detection processing step 104, the process jumps to the view field dividing step 
201 . In the case where there is an entering object detected, on the other hand, the process proceeds to the alarm/mon- 
itor indication step 106 (alarm/monitor branching step 105). In the alarm/monitor indication step 106, the alarm lamp 

so 610 is turned on or the result of the entering object detection process is indicated on the monitor 61 1 . The alarm/mon- 
itor indication step 106 is followed also by the view field dividing step 201. Means for transmitting an alarm as to the 
presence or absence of an entering object to the guardsman (or an assisting living creature, which may be the guards- 
man himself, in charge of transmitting information to the guardsman) may be any device using light, electromagnetic 
wave, static electricity, sound, vibrations or pressure which is adapted to transmit an alarm from outside of the physical 

55 body of the guardsman through any of his sense organs such as aural, visual and tactile ones, or other means giving 
rise to an excitement in the body of the guardsman. 

[0041] Now. the process of steps 201 to 205 in theflowchart of Fig. 1 will be explained with reference to Figs. 5, 7 and 
8. 
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[0042] In the view field dividing step 201 , the view field is divided into a plurality of view field areas, and the process 
proceeds to the image change detection step 202. Specifically, the process of steps 202 to 205 is repeated for each 
divided view field area. 

[0043] In the view field dividing step 201 , division of the view field is previously determined based on, for example, an 
5 average moving distance of an entering object, a moving direction thereof, for example, parallel to the moving direction 
(for example, traffic lanes when the entering object is a vehicle) or perpendicular thereto, a staying time of an entering 
object or the like. Other than these, by setting dividing border lines to border portions existing in the monitoring view 
field (for example, a median strip, a median line, a border line between roadway and sidewalk or the like when a moving 
object is a vehicle moving on a road), it becomes possible to make mismatching portions between those pixels of the 
w reference background image that are updated and those not updated harmless. Other than those dividing portions 
mentioned above, the view field may be divided at any portions that may possibly cause intensity mismatching such as 
wall, fence, hedge, river, waterway, curb, bridge, pier, handrail, railing, cliff, plumbing, window frame, counter in a lobby, 
partition, apparatuses such as ATM terminals, etc. 

[0044] The process from the image change detection step 202 to the divided view field end determination step 205 
is is executed for each of the plurality of divided view field areas. Specifically, the process of steps 202 to 205 is repeated 
for each divided view field area. First, in the image change detection step 202, a changed area existing in the input 
image is detected for each divided view field area independently. Fig. 5 is a diagram for explaining an example of the 
method of processing the image change detection step 202. In Fig. 5, numeral 1001 designates an input image at time 
point to-2, numeral 1002 an input image at time point to-1 , numeral 1003 an input image at time point to. numeral 1004 
20 a bi narized difference image obtained by determining the difference between the input image 1 002 and the input image 
1003 and binarizing the difference, numeral 1005 a binarized difference image obtained by determining the difference 
between the input image 1003 and the input image 1002 and binarizing the difference, numeral 1006 a changed area 
image, numeral 1007 an entering object detection area of the input image 1001 at time point V 2 . numeral 1008 an 
entering object detection area of the input image 1002 at time point t 0 -1. numeral 1009 an entering object detection 
25 area of the input image 1003 at time point to. numeral 1010 a detection area of the binarized difference image 1004, 
numeral 101 1 a detection area of the binarized difference image 1005, numeral 1012 a changed area, numerals 1021, 
1022 difference binarizers, and numeral 1023 a logical product calculator. 

[0045] In Fig. 5. entering objects existing in the input image 1001 at time point t 0 -2, the input image 1002 at time point 
to-1 and the input image 1003 at time point to are indicated as a model, and each entering object proceeds from right 
30 to left in the image. This image change detection method regards time point to as the present time and uses input 
images of three frames including the input image 1001 at time pint to-2, the input image 1002 at time point to-1 and the 
input image 1003 at time point to stored in the image memory 603. 

[0046] In the image change detection step 202, the difference binarizer 1 021 calculates the difference of the intensity 
or brightness value for each pixel between the input image 1001 at time point to-2 and the input image 1002 at time point 
35 to-1 , and binarizes the difference in such a manner that the intensity or brightness value of the pixels for which the dif- 
ference is not less than a predetermined threshold level (20, for example, in this embodiment) is set to w 255". while the 
intensity value of the pixels less than the predetermined threshold level is set to "0". As a result, the binarized difference 
image 1004 is produced. In this binarized difference image 1004, the entering object 1007 existing in the input image 

1001 at time point V 2 is overlapped with the entering object 1008 existing in the input image 1002 at time point to-1. 
40 and the resulting object is detected as the area (object) 1010. In similar fashion, the difference between the input image 

1002 at time point to-1 and the input image 1 003 at time point to is determined by the difference binarizer 1022 and bina- 
rized with respect to the threshold level to produce the binarized difference image 1005. In this binarized difference 
image 1005, the entering object 1008 existing in the input image 1002 at time point tQ-1 is overlapped with the entering 
object 1009 existing in the input image 1003 at time point to. and the resulting object is detected as the area (object) 

45 1011. 

[0047] Then, the logical product calculator 1023 calculates the logical product of the binarized difference images 
1004, 1005 for each pixel thereby to produce the changed area image 1006. The entering object 1008 existing at time 
point to-1 is detected as a changed area (object) 1012 in the changed area image 1006. As described above, the 
changed area 1012 with the input image 1002 changed by the presence of the entering object 1008 is detected in the 

so image change detection step 202. 

[0048] In Fig. 5, a vehicle enters or moves, and this entering or moving vehicle is produced as the changed area 1012. 
[0049] The image change detection method descrtoed with reference to Fig. 5 is disclosed in H. Ohata et al. "A 
Human Detector Based on Flexible Pattern Matching of Silhouette Projection" MVA 94 IAPR Workshop on Machine 
Vision Applications Dec. 13-15, 1994. Kawasaki, the disclosure of which is hereby incorporated by reference. 

55 [0050] At the end of the image change detection step 202, the input image 1002 at time point to-1 is copied in the area 
for storing the input image 1001 at time point V 2 * n the image memory 603, and the input image 1003 aftime point to 
is copied in the area for storing the input image 1002 at time point V 1 in *e image memory 603 thereby to replace the 
information in the storage area in preparation for the next process. After that, the process proceeds to the division 
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update process branching step 203. 

[0051] As described above, the image change between time points at which the input images of three frames are 
obtained can be detected from these input images in the image change detection step 202. As far as a temporal image 
change can be obtained, any other methods can be used with equal effect, such as by comparing the input images of 

5 two frames at time points to and V 1 - 

[0052] Also, in Fig. 6, the image memory 603, the work memory 604 and the program memory 605 are configured as 
independent units. Alternatively, the memories 603, 604, 605 may be distributed to one storage unit or a plurality of stor- 
age units, or given one of the memories may be distributed among a plurality of storage units. 
[0053] In the case where the image changed area 1 01 2 is detected in the divided view field areas to be processed, 

w by the image change detection step 202, the process branches to the divided view field end determination step 205 in 
the division update process branching step 203. In the case where the image changed area 1012 is not detected, on 
the other hand, the process branches to the reference background image update step 204. 

[0054] In the reference background image update step 204, the portion of the reference background image 702 cor- 
responding to the divided view field area to be processed by the add-up method of Fig. 8 is ipdated using the input 

15 image at time point t0-1, and the process proceeds to the divided view field area end determination step 205. In the 
reference background image update step 204, the update rate 804 can be set to a higher level than in the prior art 
because the absence of the image change in the view field area to be processed is guaranteed by the image change 
detection step 202 and the division updated process branching step 203. A high update rate involves only a small 
amount of the update processing from the time of an input image change to the time when the change is updated in the 

20 reference background image. In the case where the update rate 804 is reset from 1/64 to 1/4, for example, the update 
process can be completed only with four frames from the occurrence of an input image change to the updating in the 
reference background image. Thus the reference background image can be updated within less than one second even 
when the entering object detection process is executed at the rate of five frames per second. According to this embod- 
iment, the reference background image required for the detection of an entering object can be updated within a shorter 

25 time than in the prior art, and therefore an entering object can be positively detected even in a scene where the illumi- 
nance of the view field environment undergoes a change. 

[0055] In the divided view field end determination step 205. it is determined whether the process of the image change 
detection step 202 to the reference background image division update processing step 204 has been ended for all the 
divided view field areas. In the case where the process is not ended for all the areas, the process returns to the image 

30 change detection step 202 for repeating the process of steps 202 to 205 fa the next divided view field area. In the case 
where the process of the image change detection step 202 to the reference background image division update process- 
ing step 205 has been ended for all the divided view field areas, on the other hand, the process returns to the image 
input step 101 , and the series of process of steps 101 to 205 is started from the next image input. Of course, after the 
divided view field end determination step 205 or in the image input step 101, the process may be delayed a predeter- 

35 mined time thereby to adjust the processing time for each frame to be processed. 

[0056] In the embodiment described above, the view field is divided into a plurality of areas in the view field dividing 
step 201, and the reference background image is updated independently for each divided view field area in the refer- 
ence background image division update processing step 204. Even in the case where an image change has occurred 
in a portion of the view field, therefore, the reference background image can be updated in the divided view field areas 

40 other than the changed area. Also, it is possible to easily specify the place of mismatch of the intensity value between 
pixels updated and not updated which occurs only in the boundary of divided view field areas which are preset in the 
reference background image updated by the dynamic area updating method. As a result, the reference background 
image required for detecting entering objects can be updated within a short time, and even in a scene where the illumi- 
nance of the view field areas suddenly changes, an entering object can be accurately detected. 

45 [0057] Another embodiment of the invention will be explained with reference to Fig. 2. In this embodiment, the view 
field is divided into a plurality of areas and the entering object detection process is executed for each divided view field 
area. Fig. 2 is a flowchart for explaining the process of updating the reference background image and detecting an 
entering object according to this embodiment of the invention. In this flowchart, the view field dividing step 201 in the 
flowchart of Fig. 1 is executed before detection of an entering object, i.e. after the binarization step 103. Further, the 

so entering object detection processing step 104 is replaced by a divided view field area detection step 301 for detecting 
an entering object in each divided view field area, the alarm/monitor branching step 105 is replaced by a divided view 
field area alarm/monitor branching step 302 for determining the presence or absence of an entering object for each 
divided view field area, and the alarm/monitor incfication step 1 06 is replaced by a divided view field area alarm/monitor 
indication step 303 for issuing or indicating an alarm on a monitor for each divided view field area. The divided view field 

55 areas covered by the divided view field area detection step 301. the divided view field area alarm/monitor branching 
step 302 and the alarm/monitor indication step 303 are derived from the view field area dividing step 201 for dividing 
the view field covered by the entering object detection processing step 104, the alarm/monitor branching step 105 and 
the alarm/monitor indication step 106, respectively. 
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[0058] As described above, according to this invention, the reference background image is updated for each divided 
view field area independently, and therefore the mismatch described above can be avoided in each divided view field 
area. Also, since the brightness mismatch occurs in the known boundary of divided view field areas in the reference 
background image, an image memory of small capacity can be used and further it can be easily determined from the 

5 location of a mismatch whether pixels that are detected are caused by the mismatch or really correspond to an entering 
object, so that the mismatch poses no problem in object detection. In other words, the detection error (the error of the 
detected shape, the error in the number of detected objects, etc.) which otherwise might be caused by the intensity mis- 
match between the pixel for which the reference background image can be updated and the pixel for which the refer- 
ence background image cannot be updated can be prevented and an entering object can be accurately detected. 

io [0059] Still another embodiment of the invention will be explained with reference to Figs. 3A, 3B. Fig. 3A shows an 
example of a lane image caught in the image pickip view field of the TV camera 601, and Fig. 3B shows an example 
of division of the view field. In this example, the view field is divided based on the average direction of movement of 
entering objects measured in advance in the view field dividing step 201 of the f lowchart of Fig. 2, in which the objects 
to be detected by monitoring a road are automotive vehicles. Numeral 401 designates a view field, numeral 402 a view 

75 field area, numerals 403, 404 vehicles passing through the view field 401 , numerals 405, 406 arrows indicating the aver- 
age direction of movement, and numerals 407, 408, 409, 410 divided areas. 

[0060] In Fig. 3A, the average direction of movement of the vehicles 403, 404 passing through the view field 401 is 
as shown by arrows 405, 406. respectively. This average direction of movement can be measured in advance at the 
time of installing the image monitoring system. According to this invention, the view field is divided in parallel to the aver- 
se age direction of movement, as explained below with reference to Figs. 1 1 A to 1 1 C. Numeral 1101 designates an exam- 
ple of the view field divided into a plurality of view field areas, in which paths 1 101a, 1 101b of movement of the object 
to be detected obtained when setting the monitoring view field are indicated in overlapped relation. The time taken by 
an object entering the view field before leaving the view field along the path 1 101a is divided into a predetermined 
number (four, in this example) of equal parts, and the position of the object at each time point is expressed as a1, a2, 
25 a3. a4, a5 (the coordinate of each position is expressed by (X a1 , X a1 ) for a1 , for example). Also, the vectors of each sec- 
tion are expressed as a21 , a32, a43, a54 (anm represents a vector connecting a position an and a position am). In sim- 
ilar fashion, the time taken by an object entering the view field before leaving the view field by plotting the path 1 101b 
is divided into a predetermined number of equal parts, and the position of the object at each time point is expressed as 
b1 , b2, b3, b4, b5 (the coordinate of each position is expressed as (X^, Y b1 ) for b1 , for example). The vector of each 
30 section is given as b12, b23, b34, b45 (bnm indicates a vector connecting a position bn and a position bm). Thus, the 
vector of each section indicates the average direction of movement. Also, the intermediate points between the positions 
a1 and b1 , the positions a2 and b2, the positions a3 and b3, the positions a4 and b4 and the positions a5 and b5 are 
expressed as c1, c2, c3, c4, c5, respectively, (the coordinate of each position is expressed as (X^, Y C1 ) for d, for 
example). In other words, X c1 =(X a1 + X b1 V2, Y C1 = (Y a1 + Y b1 )/2(i: 1 to 5). The line 1102c connecting the points 
35 ci thus obtained is assumed to be a line dividing the view field (1 102). Thus, the view field is divided as shown by 1 103. 
Also, even in the case where there are not less than three routes of movement of objects, the moving paths of the 
objects following adjacent routes are determined in the same manner as in the case of Figs. 1 1 A to 1 1C. 
[0061 ] In the case of the view field 401 . for example, as shown by the view field area 402, the image pickup view field 
is divided into areas 407, 408, 409, 410 by lanes. Entering objects can be detected and the reference background 
40 image can be updated for each of the divided view field areas. Therefore, even when an entering object exists in one 
divided view field area (lane) and the reference background image of the divided view field area of the particular lane 
cannot be updated, the reference background image can be updated in other divided view field areas (lanes). Thus, 
even when an entering object is detected in a view field area, the reference background image required for the entering 
object detection process in other divided view field areas can be updated within a shorter time than in the prior art 
45 shown in Fig. 2. In this way, even on a scene where the illuminance of the view field area undergoes a change, an enter- 
ing object can be accurately detected. 

[0062] A further embodiment of the invention will be explained with reference to Figs. 4A, 4B. Fig. 4A shows an exam- 
ple of a lane image caught in the image pickup view field of the TV camera 601 , and Fig. 4B shows an example division 
of the view field). This embodiment represents an example in which the view field is divided in the view field dividing 
so step 201 of the flowchart of Fig. 2, based on the average distance coverage of an entering object measured in advance, 
and the object to be detected by monitoring a road is assumed to be a vehicle. Numeral 501 is a view field, numeral 502 
a view field area, numeral 503 a vehicle passing through the view field 501 , numeral 504 an arrow indicating the aver- 
age distance coverage, and numerals 505, 506. 507, 508 divided areas. 

[0063] In Fig. 4A. the moving path of the vehicle 503 passing through the view field 501 is indicated by arrow 504. 
55 This moving path can be measured in advance at the time of installing an image monitoring system. According to this 
invention, the view field is divided into equal parts by an average distance coverage based on object moving paths so 
that the time taken for the vehicle to pass through each divided area is constant. This will be explained with reference 
to Figs. 12 A, 12B and 12C. Numeral 1201 in Fig. 12A designates an example of the view field area to be divided, in 
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which moving paths 1201d and 1201 e of objects obtained when setting a monitoring view field are shown in overlapped 
relation. The time required for the entering object plotting the moving path 1201d before leaving the view field is divided 
into a predetermined number (four in this case) of equal parts, and the position of the object at each time point is 
expressed as d1 . d2, d3, d4, d5 (the coordinate of each position is expressed as (X^, Y d1 ) for d1 , for example). In sim- 

5 ilar fashion, the time required for the entering object plotting the moving path 1201e before leaving the view field is 
divided into a predetermined number of equal parts, and the position of the object at each time point is expressed as 
e1, e2, e3, e4, e5 (the coordinate of each position is expressed as (X e1 , Y e1 ) for e1, for example). Then, the displace- 
ment of each position represents the average moving distance range. The straight lines connecting positions d1 and 
el , positions d2 and e2, positions d3 and e3, positions d4 and e4 and positions d5 and e5 are expressed as L1 , L2, L3, 

70 L4, L5, respectively. In other words, the relation U:y = (y ei - y dj )/(x ej - x d i) x (x - x dl ) + y dj (i: 1 to 5) is assumed, 
where each value Li obtained is assumed as a line dividing the view field (1202). Thus, the view field is divided as 
shown by 1203. Also when three of more routes of movement of an object exist, a set of routes is arbitrarily selected 
and the divided areas can be determined in a manner similar to the example shown in Figs. 12A to 12C. 
[0064] In the example of the view field 501 , as shown in the view field area 502, the image pickup view field area 501 

is is divided into four areas 505, 506, 507, 508. However, the view field can be divided into other than four areas. An enter- 
ing object is detected and the reference background image is updated for each divided view field area. Thus, even in 
the case where an entering object exists in one lane, the entering object detection process can be executed in the 
divided view field areas other than the area where the entering object exists. The divided areas can be indicated by dif- 
ferent colors on the screen of the monitor 61 1 . Further, the boundaries between the divided areas may be displayed on 

20 the screen. This is of course also the case with the embodiments of Figs. 3A, 3B. 

[0065] Also, when monitoring other than a road, such as a harbor, the view field can be divided in accordance with 
the time where an object stays in a particular area where the direction or moving distance of a ship in motion can be 
specified, such as at the entrance of a port, a wharf, a canal or straits. 

[0066] As described above, even when an entering object is detected in a view field area, the reference background 
25 image required for the entering object detection process in other divided view field areas can be updated within a 
shorter time than in the prior art shown in Fig. 1. Thus, an entering object can be accurately detected even in a scene 
where the illuminance changes in a view field area. 

[0067] In other embodiments of the invention, the view field is divided by combining the average direction of move- 
ment and the average moving distance as described with reference to Figs. 3A, 3B, 4A, 4B. Specifically, in the embod- 

30 iment of Fig. 3, the reference background image cannot be updated in a particular lane where an entering object exists. 
Also, in the embodiment of Fig. 4, in the case where an entering object exists in an area or segment of the road, the 
reference background image cannot be updated for the area or segment. By dividing the view field into several lanes 
and several areas, however, the entering object detection process can be executed in divided view field areas other 
than the lane or the segment where the entering object exists. As a result, even in the case where an entering object is 

35 detected in a given view field area, the reference background image required for the entering object detection process 
can be updated in a shorter time than in the prior art in other than the divided view field area where the particular enter- 
ing object exists. In this way, an entering object can be accurately detected even in a scene where the illuminance of 
the view field environment changes. 

[0068] As described above, according to this invention, even in the case where an entering object is detected in a 
40 view field area the reference background image required for the entering object detection process in other than the 
divided view field area where the particular entering object exists can be updated in a shorter time than when updating 
the reference background image by the conventional add-up method. Further, the brightness mismatch between pixels 
that can be updated and pixels in the divided view field areas that cannot be updated can be prevented unlike in the 
conventional dynamic area updating method. Thus, an entering object can be accurately detected even in a scene 
45 where the illuminance of the view field environment undergoes a change. 

[0069] It will thus be understood from the foregoing description that according to this embodiment, the reference back- 
ground image can be updated in accordance with the brightness change of the input image within a shorter time than 
in the prior art using an image memory of a fewer capacity. Further, unlike in the prior art, the intensity mismatch 
between pixels for which the reference background image can be updated and pixels for which the reference back- 
so ground image cannot be updated is obviated by regarding them to be located at a specific place such as the boundary 
line between the divided view field areas. It is thus possible to detect only an entering object accurately and reliably, 
thereby widening the application of the entering object detecting system considerably while at the same time reducing 
the capacity of the image memory. 

[0070] The method for updating the reference background image and the method for detecting entering objects 
55 according to the invention described above can be executed as a software product such as a program realized on a 
computer readable medium. 
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Claims 

1. A method of updating a reference background image for use in detecting one or more objects entering an image 
pickup view field based on a difference between an input image and the reference background image of said input 

s image, comprising the steps of: 

dividing said view field into a plurality of view field areas (201); and 

detecting a change in a video signal of each part of said input image corresponding to each of said divided view 
field areas independently (202); and 
w updating a part of said reference background image corresponding to that part of said input image which has 

no image signal change (204). 

2. A method according to claim 1 , wherein said change includes a movement of an entering object. 

75 3. A method according to claim 1 , further comprising the step of determining whether an entering object exists or not 
independently for each of said divided view field areas (302), 

wherein the steps of detecting and updating are applied to each of those divided view field areas that are deter- 
mined to have no entering object. 

20 4. A method according to claim 1 , wherein said dividing step includes the step of dividing said image pickup view field 
by one or more boundary lines generally parallel to the direction of movement of said entering objects. 

5. A method according to claim 1 , wherein said dividing step includes the step of dividing said image pickup view field 
within the average movement range of said entering object during each predetermined unit time. 

25 

6. A method according to claim 1, wherein said dividing step includes the step of dividing said view field by one or 
more boundary lines generally parallel to the direction of movement of said entering object and further subdividing 
said divided view field in an average movement range of said entering object during each predetermined unit time. 

30 7. A method according to claim 1 , wherein said input image includes a lane, and said dividing step includes the step 
of dividing said image pickup view field by one or more said lane boundaries. 

8. A method according to claim 1 , further comprising the step of displaying the boundaries of said divided view field 
areas on a display screen. 

35 

9. A method according to claim 1 , further comprising the step of displaying said divided view field areas on a display 
screen in different colors. 

10. A method according to claim 1, wherein said input image includes at least a lane and said dividing step includes 
40 the step of dividing said image pickup view field by an average movement range of vehicles during each predeter- 
mined unit time. 

11. A method according to claim 1, wherein said input image includes at least a lane and said dividing step includes 
the step of dividing said image pickup view field by one or more lane boundaries and the step of subdividing said 

45 divided image pickup view field by an average movement range of the vehicle during each predetermined unit time. 

12. A method according to claim 1 , further comprising the step (104) of determining whether an entering object exists 
in said image pickup view field or not, said dividing step (201) and said step (204) of updating the reference back- 
ground image portion being executed after executing said determination step. 

so 

13. A method according to claim 1 , wherein said dividing step divides said image pickup view field into a plurality of 
view field areas based on at least one of the average direction of movement of said entering object and the distance 
coverage of said entering object during a predetermined unit time. 

55 1 4. A system for updating the reference background image for use in the detection of objects entering the image pickup 
view field based on the difference between an input image and the reference background image of said input 
image, comprising: 
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a dividing unit (201) for dividing said image pickup view field into a plurality of view field areas; and 

an updating unit (204) for updating the part of the reference background image corresponding to each of said 

plurality of the divided view field areas independently for each of said divided view field areas. 

5 1 5. A method according to claim 14, wherein said dividing unit (201) divides said image pickup view field into a plurality 
of view field areas based on at least one of the average direction of movement of said entering objects and the dis- 
tance covered by said entering objects for each predetermined unit time. 

16. A method according to claim 14, wherein said updating unit includes: 

10 

an image change detection unit (202) for detecting the change of the image signal of said input image corre- 
sponding to each of said divided view field areas independently for each of said divided view field areas; and 
a background image updating unit (204) for updating the part of said reference background image correspond- 
ing to each of said divided view field areas where the image signal has not changed. 

75 

17. A computer readable medium having computer readable program code means embodied therein tor detecting one 
or more entering objects in the image pickup view field based on the difference between an input image and a ref- 
erence background image of said input image, comprising: 

20 code means (201) for dividing said image pickup view field into a plurality of view field areas; and 

code means (204) for updating the part of said reference background image corresponding to each of said plu- 
rality of the divided view field areas independently for each of said divided view field areas. 

18. A method according to claim 17. wherein said updating code means (204) includes: 

25 

code means (202) for detecting the change of said input image within each of said plurality of the divided view 
field areas independently for each of said divided view field areas; and 

code means (204) for updating the part of said reference background image corresponding to each of said 
divided view field areas independently for each of said divided view field areas in which the change of said 
30 input image signal is not detected. 

19. A method of detecting entering objects, comprising the steps of: 

detecting one or more objects entering an image pickup view field based on the difference between an input 
35 image and a reference background image of said input image (1 04); 

dividing said image pickup view field into a plurality of view field areas (201); and 

updating the part of said reference background image corresponding to each of said plurality of the divided 
view field areas independently for each of said divided view field areas for detecting the next entering object 
(204). 

40 

20. A method according to claim 1 9, wherein said step of updating said reference background image includes the steps 
of: 

dividing said image pickup view field into a plurality of view field areas (201); 
45 detecting the change of the image signal of the input image portion corresponding to each of said divided view 

field areas (202) independently for each of said divided view field areas; and 

updating the portion of said reference background image corresponding to each of said divided viewf ield areas 
corresponding to said input image portion in which the image signal has not changed (204). 

so 21. A method according to claim 19. wherein said change of said image signal is the movement of an entering object. 

22. A method according to claim 19. wherein said dividing step includes the step of dividing said image pickup view 
field by one or more boundary lines generally parallel to the direction of movement of said entering object. 

55 23. A method according to claim 19, wherein said dividing step includes the step of dividing said image pickup view 
field by an average movement range of said entering object during a predetermined unit time. 

24. A method according to claim 19, wherein said dividing step includes the step of dividing said view field by one or 
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more boundary lines generally parallel to the direction of movement of said entering object and subdividing said 
divided view field by an average movement range of said entering object during each predetermined unit time. 

25. A method according to claim 19, wherein said input image includes at least a lane and said dividing step includes 
5 the step of dividing said image pickup view field by one or more lane boundaries. 

26. A method according to claim 19. wherein said input image includes a lane, and said dividing step includes the step 
of dividing said image pickup view field by an average movement range of a vehicle during a predetermined unit 
time. 

10 

27. A method according to claim 19, wherein said input image includes a lane and said dividing step includes the step 
of dividing said image pickup view field by one or more lane boundaries and subdividing said divided image pickup 
view field by an average movement range of the vehicle during a predetermined unit time. 

15 28. A method according to claim 19, wherein said dividing step divides said image pickup view field into a plurality of 
view field areas based on at least one of the average direction of movement of said entering object and the distance 
covered by said entering object during a predetermined unit time. 

29. A method of detecting entering objects, comprising the steps of: 

20 

dividing said image pickup view field into a plurality of view field areas (201); and 

detecting one or more objects entering an image pickup view field based on the difference between an input 
image and the reference background image of said input image (104) and updating said reference background 
image for detecting the next entering object (204). 

25 

30. A method according to claim 29, wherein said step of updating said reference background image includes the steps 
of: 

detecting the change of the image signal of the input signal corresponding to each of said divided view field 
30 areas (202); and 

updating said reference background image corresponding to said divided view field area in the absence of the 
change of the image signal (204). 

31. A method according to claim 29, wherein said dividing step includes the step of dividing said image pickup view 
35 field by the average movement range of said entering object during each predetermined unit time. 

32. A method according to claim 29, wherein said dividing step includes the steps of dividing said view field by one or 
more boundary lines generally parallel to the direction of movement of said entering object and subdividing said 
divided view field areas by the average movement range of said entering object during each predetermined unit 

40 time. 

33. A method according to claim 29, wherein said input image includes at least a lane and said dividing step includes 
the step of dividing said image pickup view field by one or more lane boundaries. 

45 34. A method according to claim 29, wherein said input image includes at least a lane, and said dividing step includes 
the step of dividing said image pickup view field by the average movement range of a vehicle during a predeter- 
mined unit time. 

35. A method according to claim 29, wherein said input image includes at least a lane, and said dividing step includes 
so the step of dividing said image pickup view field by one or more lane boundaries and subdividing said divided 

image pickup view fields by the average movement range of a vehicle during each predetermined unit time. 

36. A system for detecting entering objects, comprising: 

55 an image input unit (601 , 602); and 

a processing unit including an image memory (603) for storing the input image from said image input unit, a 
program memory (606) for storing the program for operation of said entering object detecting unit, and a central 
processing unit (605) for activating said entering object detecting unit in accordance with said program for 
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processing said input image; 
said processing unit including: 

an entering object detecting unit (102 to 104; 102, 103, 301) for determining the difference for each pixel 
between said input image from said image input unit and the reference background image not including an 
entering object to be detected and detecting an area with said difference larger than a predetermined threshold 
value as an entering object; 

a dividing unit (201) for dividing the image pickup view field of said image input unit into a plurality of view field 
areas; 

an image change detecting unit (202) for detecting the change of the image in each of said divided view field 
areas; and 

a reference background image updating unit (204) for updating each portion of said reference background 
image corresponding to that divided view field area corresponding to the portion of said input image of which 
the image has not changed; 

wherein said entering object detecting unit detects an entering object based on said updated reference back- 
ground image. 

37. A computer readable medium having computer readable program code means embodied therein for detecting 
entering objects in an image pickup view field, comprising: 

code means (102 to 103) for detecting the intensity difference for each pixel between an input image and the 
present reference background image of said input image; 

code means (104) for detecting one or more entering objects in the image pickup view field based on said 
intensity difference; and 

code means (204) for updating said present reference background image for detecting the intensity difference 

for each pixel between a newly input image and said reference background image; 

wherein said code means for updating said reference background image includes: 

code means (201) for dividing said image pickup view field into a plurality of view field areas; 

code means (202) for detecting the change of the image signal of the input image portion corresponding to 

each of said plurality of said divided view field areas independently for each of said divided view field areas; 

and 

code means (204) for updating the portion of said reference background image corresponding to each of said 
divided view field areas corresponding to the portion of said input image of which said image signal has not 
changed. 

38. A computer readable medium having computer readable program code means embodied therein for detecting 
entering objects in an image pickup view field, comprising: 

code means (201) for dividing the image pickup view field of an image pickup device into a plurality of view field 
areas; 

code means (301) for detecting one or more objects entering said divided view field areas based on the inten- 
sity difference for each pixel between the image input from said image pickup device and the reference back- 
ground image independently for each of said divided view fields; and 

code means (204) for updating said reference background image corresponding to each of said divided view 
fields tor detecting the intensity difference between a newly input image and said reference background image 
independently for each of said divided view field areas. 

39. A method according to claim 38, wherein said code means for updating said partial reference background image 
includes: 

code means (202) for detecting the change of the image signal of the input image corresponding to each of 
said divided view field areas; and 

code means (204) for updating said reference background image corresponding to each of said divided view 
field areas in the absence of the change of said image signal. 
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